Development of Pen-based Note-taking System for Persons with Visually Disabilities

We frequently take notes when attending class. The method of note-taking includes writing much information as a paper record. However, we cannot write such notes in a notebook if no visual feedback information exists. We propose a pen-based note-taking system, designated as the Pen-Talker, for a visually disabled person to use on an ultra-mobile PC. Using the Pen-Talker, a blind novice user with acquired blindness can input Japanese characters directly instead of through a keyboard without much training. This paper describes the system design concept. We also investigate a higher level recognition engine based on hybrid recognition, which integrates on-line and off-line recognition algorithms; we restructure the input interface using the ultra mobile PC with a voice-assistance function. Screen information is given to users via a built-in a screen reader. Therefore, the Pen-Talker is useful as note pad software using a button operation that is simple for even a blind novice user. Recognition experiment results show that recognition accuracy has improved to 93.3% for eight blind subjects, which is an extremely high score compared to that of our previous system. Results confirmed the possibility of practical use of the proposed system.